Arginine vasotocin neuronal phenotypes, telencephalic fiber varicosities, and social behavior in butterflyfishes (Chaetodontidae): potential similarities to birds and mammals.

The neuropeptide arginine vasopressin (AVP) influences many social behaviors through its action in the forebrain of mammals. However, the function of the homologous arginine vasotocin (AVT) in the forebrain of fishes, specifically the telencephalon remains unresolved. We tested whether the density of AVT-immunoreactive (-ir) fiber varicosities, somata size or number of AVT-ir neuronal phenotypes within the forebrain were predictive of social behavior in reproductive males of seven species of butterflyfishes (family Chaetodontidae) in four phylogenetic clades. Similar to other fishes, the aggressive (often territorial) species in most cases had larger AVT-ir cells within the gigantocellular preoptic cell group. Linear discriminant function analyses demonstrated that the density of AVT-ir varicosities within homologous telencephalic nuclei to those important for social behavior in mammals and birds were predictive of aggressive behavior, social affiliations, and mating system. Of note, the density of AVT-ir varicosities within the ventral nucleus of the ventral telencephalon, thought to be homologous to the septum of other vertebrates, was the strongest predictor of aggressive behavior, social affiliation, and mating system. These results are consistent with the postulate that AVT within the telencephalon of fishes plays an important role in social behavior and may function in a similar manner to that of AVT/AVP in birds and mammals despite having cell populations solely within the preoptic area.